Quantifying the output of credit research systems

ABSTRACT

A method of quantifying the value added by an internal company credit rating system is described. The method includes determining an internal company credit rating for a plurality of securities. The internal company credit rating for the plurality of securities and an external credit research agency original credit rating for each of the securities are inputted in a ratings history database. Data representative of a change of the external credit research agency original credit rating for at least one security of the plurality of securities to a new credit rating is received. Responsive to the change, data representative of a current price of the at least one security and a benchmark price of the at least one security is received. At least one metric is calculated to determine a correlation between the internal company credit ratings and the new external credit research agency for each of the plurality of securities.

This application is a continuation of U.S. application Ser. No.11/950,375 filed Dec. 4, 2007, now allowed, the contents of which areincorporated by reference in their entirety herein.

BACKGROUND

For continued growth and development of financial assets, companiesutilize credit research systems to determine the applicable securitiesto invest in for the company. Cash investment credit research processesaddresses short term securities distinguishing versus longer term bonds,that are usually less than a year in maturity, fixed income instruments.Credit research systems evaluate credit ratings for securities that acompany then utilizes for determining what to invest in, and how much toinvest.

For any security to invest in, the pricing of the security and the yieldof the security are highly dependent on the credit quality of the issuerof the security. As such, an issuer of a security with a lower creditquality rating, i.e., a higher risk, equates to the security having ahigher return due to the higher risk involved. Companies utilize thisinformation to invest more or less in different securities.

When at all possible, goals for investments on a security for a companyinclude lowering risk and increasing return. With respect to the creditrating of an issuer of a security, 1 or 2 basis points may be a hugedifferentiator.

Currently there is no mechanism to assess objectively the performance ofa credit rating system at the aggregate, sector, and/or analyst levels.A robust credit rating process is a prerequisite for providing high riskadjusted returns to investors and ensuring capital preservation.Currently there are no metrics in place to measure a ratingeffectiveness. No formal process exists to monitor the timing andsequence of internal ratings, agency ratings, and market reaction tothose ratings.

Therefore, there exists a need in the art for a system and method thatquantifies the value added by a cash investment credit research agency.

SUMMARY

In light of the foregoing background, the following presents asimplified summary of the present disclosure in order to provide a basicunderstanding of some aspects of the disclosure. This summary is not anextensive overview of the disclosure. It is not intended to identify keyor critical elements of the disclosure or to delineate the scope of thedisclosure. The following summary merely presents some concepts of thedisclosure in a simplified form as a prelude to the more detaileddescription provided below.

Aspects of the present disclosure are directed to a method and systemfor a new cash investment credit research process that quantifies thevalue added by a cash investment credit research team. The new processfacilitates marketing credit capabilities as a competitivedifferentiator, and associating the credit scores to evaluate therelative performance and compensation of credit research analysts. Thenew process ultimately results in better risk adjusted returns toinvestors, preservation of capital, and the capability to anticipaterisks or opportunities ahead of external credit rating agencies.

In accordance with other aspects of the present disclosure, a method ofquantifying the value added by an internal company credit rating systemis described. The method includes determining an internal company creditrating for each of a plurality of securities. The internal companycredit rating for each of the plurality of securities and an externalcredit research agency original credit rating for each of the securitiesare inputted in a ratings history database. Data representative of achange of the external credit research agency original credit rating forat least one security of the plurality of securities to a new creditrating is received. Responsive to the change, data representative of acurrent price of the at least one security and a benchmark price of theat least one security is received. At least one metric is calculated todetermine the number of times the internal company credit ratings foreach of the plurality of securities was correct.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. The Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of aspects of the present disclosure andthe advantages thereof may be acquired by referring to the followingdescription in consideration of the accompanying drawings, in which likereference numbers indicate like features, and wherein:

FIG. 1 illustrates a schematic diagram of a general-purpose digitalcomputing environment in which certain aspects of the present disclosuremay be implemented;

FIG. 2 is an illustrative block diagram of workstations and servers thatmay be used to implement the processes and functions of certainembodiments of the present disclosure;

FIG. 3 is an example flow chart of an illustrative method forquantifying the value added by a cash investment credit research team inaccordance with at least one aspect of the present disclosure; and

FIG. 4 is another example flow chart of an illustrative method forquantifying the value added by a cash investment credit research team inaccordance with at least one aspect of the present disclosure.

DETAILED DESCRIPTION

In the following description of the various embodiments, reference ismade to the accompanying drawings, which form a part hereof, and inwhich is shown by way of illustration various embodiments in which thedisclosure may be practiced. It is to be understood that otherembodiments may be utilized and structural and functional modificationsmay be made.

FIG. 1 illustrates a block diagram of a generic computing device 101(e.g., a computer server) that may be used according to an illustrativeembodiment of the disclosure. The computer server 101 may have aprocessor 103 for controlling overall operation of the server and itsassociated components, including RAM 105, ROM 107, input/output module109, and memory 115.

I/O 109 may include a microphone, keypad, touch screen, and/or stylusthrough which a user of device 101 may provide input, and may alsoinclude one or more of a speaker for providing audio output and a videodisplay device for providing textual, audiovisual and/or graphicaloutput. Software may be stored within memory 115 and/or storage toprovide instructions to processor 103 for enabling server 101 to performvarious functions. For example, memory 115 may store software used bythe server 101, such as an operating system 117, application programs119, and an associated database 121. Alternatively, some or all ofserver 101 computer executable instructions may be embodied in hardwareor firmware (not shown). As described in detail below, the database 121may provide centralized storage of account information and accountholder information for the entire business, allowing interoperabilitybetween different elements of the business residing at differentphysical locations.

The server 110 may operate in a networked environment supportingconnections to one or more remote computers, such as terminals 141 and151. The terminals 141 and 151 may be personal computers or servers thatinclude many or all of the elements described above relative to theserver 101. The network connections depicted in FIG. 1 include a localarea network (LAN) 125 and a wide area network (WAN) 129, but may alsoinclude other networks. When used in a LAN networking environment, thecomputer 101 is connected to the LAN 125 through a network interface oradapter 123. When used in a WAN networking environment, the server 101may include a modem 127 or other means for establishing communicationsover the WAN 129, such as the Internet 131. It will be appreciated thatthe network connections shown are illustrative and other means ofestablishing a communications link between the computers may be used.The existence of any of various well-known protocols such as TCP/IP,Ethernet, FTP, HTTP and the like is presumed, and the system can beoperated in a client-server configuration to permit a user to retrieveweb pages from a web-based server. Any of various conventional webbrowsers can be used to display and manipulate data on web pages.

Additionally, an application program 119 used by the server 101according to an illustrative embodiment of the disclosure may includecomputer executable instructions for invoking user functionality relatedto communication, such as email, short message service (SMS), and voiceinput and speech recognition applications.

Computing device 101 and/or terminals 141 or 151 may also be mobileterminals including various other components, such as a battery,speaker, and antennas (not shown).

The disclosure is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the disclosure include,but are not limited to, personal computers, server computers, hand-heldor laptop devices, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, and the like.

The disclosure may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Thedisclosure may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

Referring to FIG. 2, an illustrative system 200 for implementing methodsaccording to the present disclosure is shown. As illustrated, system 200may include one or more workstations 201. Workstations 201 may be localor remote, and are connected by one or communications links 202 tocomputer network 203 that is linked via communications links 205 toserver 204. In system 200, server 204 may be any suitable server,processor, computer, or data processing device, or combination of thesame. Server 204 may be used to process the instructions received from,and the transactions entered into by, one or more participants.

Computer network 203 may be any suitable computer network including theInternet, an intranet, a wide-area network (WAN), a local-area network(LAN), a wireless network, a digital subscriber line (DSL) network, aframe relay network, an asynchronous transfer mode (ATM) network, avirtual private network (VPN), or any combination of any of the same.Communications links 202 and 205 may be any communications linkssuitable for communicating between workstations 201 and server 204, suchas network links, dial-up links, wireless links, hard-wired links, etc.

As understood by those skilled in the art, the steps that follow in theFigures may be implemented by one or more of the components in FIGS. 1and 2 and/or other components, including other computing devices.

Aspects of the present disclosure provide a method and an apparatus asoutlined below to compare at least two credit research systems. Aspectsinclude metrics to quantify the performance of a cash investment creditresearch system. Such metrics may be designed to work well under wideranging scenarios and business conditions.

Aspects include a robust process to collect the data needed to computethe above metrics. Data comprises of the rating changes of fixed incomesecurities and their pricing indications. The process ensures timely andaccurate acquisition of data. The process puts in place a controlmechanism to detect errors.

Further aspects of the disclosure provide an algorithm needed to computethe metrics. Aspects also provide the mapping of cash investment creditresearch system to an external agency's credit research systems. Furtheraspects of the present disclosure suggest modifications to an existingsystem to persistently store rating and pricing data and retrieve it. Inaddition, further aspects of the present disclosure provide a scorecardto illustrate the performance of a cash investment credit researchsystem.

The metrics and algorithms encapsulate more than a billion possiblescenarios. The scenarios represent all possible combinations of theratings, watch and/or outlook provided by external credit ratingagencies, such as S&P, Moody's, and Fitch, and the internal ratings of acompany. A combination, for example, may be:

S&P—Upgrade

Moody's—No change

Fitch—Downgrade

Internal Rating of Company—No change

The processes and systems put in place ensure that timely, accurate, andrelevant information is disseminated to all stakeholders. Moreover theprocesses and systems have control mechanisms in place to preventintentional/unintentional data errors, gaming of metrics, and to detectchange in organizational behavior arising from the metrics.

One metric seeks to measure the number of times a rating was right. Itis based on the following concept from probability theory. Any test canhave four (4) possible outcomes:

True positive—An internal company rating value for a security ispredicted correctly in an upgrade direction.

True negative—An internal company rating value for a security ispredicted correctly in a downgrade direction.

False positive—An internal company rating value for a security ispredicted incorrectly as the rating value moves up in grading when theinternal rating of the company expected the rating value to move in adowngrade direction.

False negative—An internal company rating value for a security ispredicted incorrectly as the rating value moves down when the internalrating of the company expected the rating value to move in an upgrade orno change direction.

Based upon this information, the effectiveness of a test may be gaugedas follows.

Effectiveness=The number of securities with spread movement in the rightdirection divided by the number of securities reviewed

-   -   =(# of true positives+# of true negatives)/(# of true        positives+# of true negatives+# of false positives+# of false        negatives)

A business objective of the first metric may be to rate the performanceof an analyst with respect to assessing the performance of the internalcompany credit rating on a security. For example, if an analyst upgradesa security and the spread reduces, the analyst was correct in upgradingthe security rating. If the analyst upgrades a security and the spreadincreases, the analyst was incorrect in upgrading the security rating.If the analyst does not change the security rating and the spreadincreases or decreases with an agency rating change that diverged fromthe internal company rating, she was incorrect in not changing, e.g.,upgrading or downgrading, the security rating.

The spread movement may be calculated as the difference between thespread as of the last rating change and the current spread. The spreadis measured with respect to a benchmark. For calculation purposes, thesecurity and benchmark prices as of a last change and as of now areneeded. In addition, a security rating history for the period inquestion is needed.

A second metric seeks to measure the number of times the companypredicted ahead of one or more of the outside credit research agencies.The percentage of times ahead is equal to the number of convergedsecurities ahead of time divided by the total number of securitiesreviewed.

A business objective of the second metric may be to rate the performanceof an analyst with respect to assessing the performance of the internalcompany credit rating on a security. For example, if the difference ofopinion between the company's internal rating for a security and therating by one or more credit research agencies reduced because the oneor more credit research agencies followed the analyst's internal companycredit rating or if the difference of opinion remained the same, ananalyst may receive credit for a correct assessment of the credit ratingon the security.

Illustrative examples follow. In a first example, assume the internalrating for a security was AAA previously. This time, the internal ratingassessment by the analyst remains unchanged as AAA. If one or more ofthe credit research agencies upgraded their rating on the security fromBBB to AAA, the analyst is ahead and may receive compensation/reward foran accurate assessment.

In a second example, assume the internal rating for a security was BBBpreviously. This time, the internal rating assessment by the analyst isto upgrade the security rating to A. If one or more of the creditresearch agencies previously rated the security as a B, and nowdowngrades the rating to C, the opinion differences previously noted donot converge and the analyst may not receive compensation/reward for aninaccurate assessment.

An analyst may receive compensation/reward if the difference of opinionbetween the rating of the internal company and one or more of the creditresearch agencies remains constant. An analyst may not receivecompensation/reward if the analyst follows the credit research agencies.For calculation purposes, the security rating history for the internalcompany and one or more of credit research agencies for a period inquestion is required.

A third metric seeks to measure the number of times the companypredicted behind of one or more of the outside credit research agencies.The percentage of times behind is equal to the number of convergedsecurities behind the time divided by the total number of securitiesreviewed.

A business objective of the third metric may be to rate the performanceof an analyst with respect to assessing the performance of the internalcompany credit rating on a security. For example, if the difference ofopinion between the company's internal rating for a security and therating by one or more credit research agencies increased because the oneor more credit research agencies proceeded against the analyst'sinternal company credit rating, an analyst may receive credit for anincorrect assessment of the credit rating on the security.

An illustrative example follows. Assume the internal rating for asecurity was AAA previously. This time, the internal rating assessmentby the analyst remains unchanged as AAA. If one or more of the creditresearch agencies downgraded their rating on the security from BBB toCCC, the analyst is behind and may receive no compensation/reward for aninaccurate assessment.

An analyst may not receive compensation/reward if the difference ofopinion between the rating of the internal company and one or more ofthe credit research agencies remains constant. For calculation purposes,the security rating history for the internal company and one or more ofcredit research agencies for a period in question is required.

A fourth metric seeks to measure the impact of a company's internalcredit ratings in terms of change in spread. The metric is based oninterest rate spread of a security with respect to a benchmark security.Interest rate comprises α (default risk premium), β (market riskpremium), and γ (income tax premium).

For calculation purposes, I1=Interest rate for the security being rated.I1=α1+β1+1+γ1, I2=Interest rate for the benchmark security. I2=α2+β2+γ2.If we choose the benchmark security such that α2=0, β2=β1, and γ2=γ1,the Spread=I1−I2=α1=the default risk premium of the security beingrated. Thus by choosing an appropriate benchmark security, the defaultrisk may be deduced by measuring the spread.

A business objective of the fourth metric may be to rate the performanceof an analyst with respect to assessing the performance of the internalcompany credit rating on a security. For example, assume the spread of asecurity with respect to a benchmark was 25 bps. Now assume the securityis upgraded, and the new spread is 20 bps. The net impact due to theupgrade is 5 bps. For calculation purposes, the security price historyis required.

In accordance with one or more aspects of the present disclosure, graphsmay be generated from calculated data to reflect rating trends. Thesemay be graphs depicting one or more credit research agency ratings withrespect to time, company internal ratings with respect to time, anddifferences of opinions between the one or more credit research agenciesratings and the company internal ratings with respect to time.

A fifth metric seeks to give the number of times a company's internalratings were different from those of one or more credit researchagencies. The percentage of times different is equal to the number ofsecurities for which different ratings exist divided by the total numberof securities reviewed. This is the number of opinion differencesbetween a company and one or more credit research agencies expressed asa percentage of the total securities reviewed.

Similar to the other metrics, a business objective of the fifth metricmay be to rate the performance of an analyst with respect to assessingthe performance of the internal company credit rating on a security. Forexample, assume that a company's internal rating on a security was BBBwhile the rating by a credit research agency was also BBB. In such anexample, the security rating was the same, i.e., no difference ofopinion. If the company's internal rating was BBB and the creditresearch agency's rating was AAA, the ratings were different. Similarly,if the company's internal rating was BBB and the credit researchagency's rating was AA, the ratings were different. For calculationpurposes, the rating history on a security by the company and the one ormore credit research agencies for a period in question is required.

FIG. 3 illustrates a high level overview of a process in accordance withat least one aspect of the present disclosure. When one or more of thecredit research agencies and/or the internal company issues a ratingchange in step 301, it may be communicated to a mailing list and ananalyst then may record the rating change in a database in step 303. Theanalyst then may send a pricing sheet to a trader to request pricing ofa security and its benchmark starting two days before the rating changeoccurred. The trader continues to price the security and its benchmarkfor ten days or until the security matures, whichever comes earlier. Anyof a number of algorithms for pricing securities may be utilized. Thealgorithm also may include a pricing mechanism for rejected securities.The analyst may store the net impact value in a database against thesecurity in question in step 305. Finally, the analyst imports the datainto a scorecard spreadsheet, runs the calculations in step 307, andcreates the scorecard in step 309. Appropriate portions of the scorecardmay be distributed to various stakeholders and others.

FIG. 4 is another example flow chart of an illustrative method forquantifying the value added by a cash investment credit research team inaccordance with at least one aspect of the present disclosure. FIG. 4illustrates sub steps that may be performed at various portions of thesteps of FIG. 3. As should be understood by one of ordinary skill, inthe illustrative example of FIG. 4, the steps of FIG. 3, steps 301-309,are illustrated by broken line boxes surrounding various steps withinFIG. 4.

With respect to FIG. 4, steps 401, 402, 403, 404, and 405 correlate tostep 301, a change in rating or outlook occurs, from FIG. 3. Steps 401,403, and 405 correlate to an example of a ratings change occurring to asecurity when done by an external credit research agency. Steps 402 and404 correlate to an example of a ratings change occurring to a securitywhen done internally as part of an internal evaluation by a company inaccordance with one or more aspects of the present disclosure.

In the example of steps 401, 403, and 405, for a ratings change by anexternal credit research agency, a master account may be used at eachcredit research agency to track issuers and/or Committee on UniformSecurity Identification Procedures (CUSIPs) products. The master accounton each credit research agency website may be set up to send an email toan account associated with a company in the event of a ratings change bythat credit research agency. Each analyst of the company may beresponsible for ensuring each credit research agency website has all ofthe issuers/CUSIPS that need to be tracked. An analyst may provide alist of names to track and another analyst may update the websites on aperiodic basis, such as quarterly. Each analyst may be responsible forchecking an inbox of the account that the credit research agency emailsto on a daily basis to check for ratings changes. In this example, instep 401, an email is received form a credit research agency. At step403, a notification, such as an email, is received in a central emailinbox of by an identified email recipient. Then, at step 405, analystsand/or traders check the boxes or receives some notification for thechange by the credit research agency to a rating/outlook on a security.

In the example of steps 402 and 404, for a ratings change internallyconducted by the company, an analyst may determine to change a rating ofa security or reject the security all together in step 402. In step 404,the analyst then may send out a credit memorandum, such as an emailand/or a report, noting the change in the rating of the particularsecurity and/or the date of the change of the rating. As should beunderstood by those skilled in the art, other data may also be includedin such a memorandum and that the rating change and date information isbut illustrative examples.

With respect to FIG. 4, steps 407 and 409 correlate to step 303, arating/outlook change being input into a database, from FIG. 3. Withrespect to steps 407 and 409, a database may be utilized. Such adatabase may be a Microsoft Access application by Microsoft Corporationof Redmond, Wash. The database is configured to maintain multipleratings changes by each external credit research agency and the internalratings by a company, marks related to certain CUSIPs, and the trackingof both taxable and municipal approved names within the same data table.

In accordance with at least one aspect of the present disclosure, aratings history table is maintained in the database. Each time a ratingis changed (such as outlook, short-term, long-term, company internal), anew record may be created within the ratings history table. As in step407, this information may be inputted by the analyst. The ratingshistory table also may take a snapshot of the unchanged ratings at thetime of the change to use in the calculation of the various metricsdescribed above. In one example, those values may be viewed by a userdouble-clicking on the change record. The analyst also may see that thedrop down options within the ratings categories may be limited, such asin step 409. The analyst may be restricted from filling in a value in anentry field other than those included in the drop down box forconsistency purposes. Further, the outlook default may be “NR”, theshort-term and long-term default may be “NR” and the company's internalrating default may be “0”. The analyst may need to ensure that all thecurrent ratings and outlooks and changes entered into the ratingshistory table are accurate. A pop-up box may be incorporated into theapproved list screen to remind the analyst to verify ratings if any arechanged. A change from “NR” may be captured in the ratings history as atime stamp of an initial rating, but it may not show an indication of“Upgrade” or “Downgrade.” All other changes may show whether the ratingschanges were up or down.

With respect to FIG. 4, steps 411, 413, and 415 correlate to step 305,pricing information being input into a database, from FIG. 3. Withrespect to steps 411, 413, and 415, the database may be the samedatabase as noted above with respect to steps 407 and 409. With respectto step 411, upon occurrence of a ratings change, an analyst responsiblefor the affected security(ies) may notify someone by sending a pricingsheet via email or other communication. The email communication andpricing sheet may note the issuer or CUSIP on the appropriate sheet forthe pricing that is requested. A trader may be notified to obtainpricing information.

With respect to step 413, the trader may provide pricing to the analystof the securities in the portfolio in addition to securities that havebeen rejected. The trader may select the securities and benchmark toprice in accordance with guidelines noted on the pricing sheet. Thetrader may price indications on selected issuers or CUSIPs from one ormore dealers that the trader reasonably believes to actively trade insuch paper. In one example, the best indication may be recorded for thatday. In cases of illiquidity or difficulty in obtaining bids, a bestefforts approach must be used. A trade may be responsible for obtainingthe price of the security(ies) and benchmark at a designated periodbefore the ratings change, such as two days before the ratings change.This information provides the “base case” pricing for the security andbenchmark. The trader may populate the pricing sheet provided by theanalyst, which may includes daily pricing and a graphical presentationof prices. The trader may record the indication daily and this may beused in the net impact metric calculation noted above. The trader alsomay be responsible for obtaining the price of the security(ies) andbenchmark in the manner described above each day for 2 days prior to thechange and then a predefined period after the ratings change, such as 10business days after the ratings change. Daily pricing may continue untilthe earliest of the following: the security(ies) mature(s) or 10 days oranother time period as agreed to by all interested parties.

With respect to securities that are rejected, the day the security isrejected, a bid side price may be recorded. Thereafter, a price may besought on all rejected securities at a designated time, such assemi-annually. Use the same benchmark and issuer characteristics aslisted above may be utilized.

The trader, the analyst, and/or one or more other individuals may berequired to enter additional data into the pricing sheet. For example,with respect to asset backed commercial paper (ABCP), the issuer's 90day cp may be used as the indication of where the issuers' paper istrading. The responsible individual may get bid side discountindications of the issuers' paper in 90 days from at least two brokersand record the best bid in an ABCP sheet for that day. The provider ofan ABCP benchmark may be the Federal Reserve. This data is offer sidediscount indications at a discount. This may posted every day on thewebsite of the Federal Reserve.

With respect to corporate securities, the 1 year maturity may be used asan indication of where the corporate issuer is trading. The responsibleindividual may get at least two bid side indications on a 1 year bondand record the spread to 1 year LIBOR index in an issuer spread column.In addition, credit default swaps off may be used as a proxy of wherethe issuer is trading. The benchmark may be determined by industry,rating, and tenor. This data may be provided as a bid side spread toLIBOR index and may be collected as a spread in the pricing spreadsheet.

With respect to asset backed securities (ABS), the pricing on an ABSsecurity may be specific to a CUSIP. The responsible individual may getat least two bid side indications and record the best bid in terms ofspread to LIBOR index. Benchmark offer side spread may be obtained froman outside company, such as JP Morgan Query. This data may be providedas a spread to LIBOR index and may be recorded as a spread in thepricing sheet.

With respect to municipal bonds, the 1 year maturity may be used as aproxy of where the issuers' paper trades. The responsible individual mayget two bid side indications and record the best bid. If a 1 year bonddoes not exist, the level may be interpolated by using a % of EDSF. Atemplate to interpolate a 1 year level may be included within thepricing sheet. Benchmark data may be obtained from an outside company,such as the Bloomberg General Obligation AA+ index, for municipalissuers. This may chosen instead of a benchmark of a specific rating andindustry since all other indexes may be too granular and may not captureenough spread to the issuer. This indication is a mid level that is theaverage of bid/side.

With respect to step 415, the data is inputted by the analyst into thedatabase. The trader or other individual provides the analyst withpricing information and net impact value data as described above and theanalyst puts the net impact value and other data into the database.

With respect to FIG. 4, steps 417, 419, and 421 correlate to step 307,run metric calculations, from FIG. 3. Data used to calculate the metricsmay be sourced from the database. In one example, a download of data,via a query in step 417, from the database may be used to capturerelevant pieces of raw data for metric calculations. The data may beexported in step 419 and calculations for each metric may be run in step421.

With respect to the first metric, ratings effectiveness, the calculationoutputs may show percentages of securities with spread movement in theright direction subsequent to a ratings change. The calculation of theratings effectiveness may be as follows. A count of securities with aratings change during a measurement period is determined. The count maybe split between securities with a ratings change internally by thecompany and those with a ratings change by one or more external creditresearch agencies. Of those securities, a count of securities with|spread movement|>0 may be determined.

If |spread movement|>0, a count of securities that have gone in theright direction following a ratings change internally by the company maybe determined. If the internal company rating is an upgrade, rightdirection means a negative spread movement. If the internal companyrating is a downgrade, right direction means a positive spread movement.If |spread movement|>0, a count of securities in which there was not aninternal company ratings change but a rating change by an externalcredit research agency may be determined. If a count of securities inwhich there was not an internal company ratings change but a ratingchange by an external credit research agency exists, a count ofsecurities with spread movement that was in the wrong direction from theexternal credit research agency ratings change perspective may bedetermined. As such, this is the number of times the company was in theright direction.

Then, the count of securities that have gone in the right directionfollowing a ratings change internally by the company is added to thecount of securities with spread movement that was in the wrong directionfrom the external credit research agency ratings change perspective.This total count then is divided by the sum the count of securities witha ratings change internally by the company and the count of securitiesin which there was not an internal company ratings change but a ratingchange by an external credit research agency. This percentage is therating effectiveness. This percentage is interpreted as the percentageof securities that have experienced spread movement in the rightdirection following an internal company's ratings change or spreadmovement in the right direction, (i.e., wrong direction from theperspective of the external credit research agencies) following aratings change by one or more of the external credit research agency. Inaddition, a percentage of securities with spread movement in the wrongdirection or flat direction alternatively may be calculated inaccordance with aspects of the present disclosure.

With respect to the second metric, percentage ahead or behind, thecalculation outputs may show a percentage of time the company is aheadof or behind one or more external credit research agencies. Thepercentage may be measured based on convergence or divergence of ratingsfollowing a credit research agency or internal company ratings change.Percentage of times ahead equate to a count of converged securitiesdivided by a count of securities with ratings changes. Percentage oftimes behind equates to a count of diverged securities divided by acount of securities with ratings changes. Percentage of times with nochange equates to a count of securities with no ratings changes dividedby a count of securities reviewed. Percentage of times with changeequates to a count of securities with a ratings change divided by acount of securities reviewed. The percentage of times ahead andpercentage of times behind sum to 1. In addition, the percentage oftimes with no change and percentage of times with change sum to 1.

The metric looks at the absolute value of differences between ratingsbefore and after a ratings change. Each ratings change is a separateevent. For example, if three external credit research agencies downgradeboth the long and short term rating for a security, the metric addressessix total observations, one for each external credit research agency foreach of the lone term rating change and the short term rating change.The internal company ratings are compared to each event. Aggregation ofthe rating change of multiple credit research agencies may be anadditional metric if desired.

If |internal company rating−rating of credit research agency|>|internalcompany rating−prior credit research agency rating|, (i.e., the ratingshave diverged), the internal company is behind the credit researchagency for that security. For example, if an internal company rating fora security is 3, a credit research agency rating is 4, and a priorrating by the credit research agency was 3, the result is 1>0, i.e., adivergence. Therefore, the internal company is behind the creditresearch agency for that security.

If |internal company rating−rating of credit research agency|<|internalcompany rating−prior credit research agency rating|, (i.e., the ratingshave converged), the internal company is ahead of the credit researchagency for that security. For example, if an internal company rating fora security is 3, a credit research agency rating is 3, and a priorrating by the credit research agency was 4, the result is 1<0, i.e., aconvergence. Therefore, the internal company is ahead of the creditresearch agency for that security.

If |internal company rating−rating of credit research agency|<|priorinternal company rating−rating of credit research agency|, (i.e., theratings have converged), the internal company is ahead of the creditresearch agency for that security. For example, if an internal companyrating for a security is 3, a credit research agency rating is 3, and aprior rating by the internal company was 4, the result is 0<1, i.e., aconvergence. Therefore, the internal company is ahead of the creditresearch agency for that security.

To measure changes, ratings may be converted to numeric values asfollows:

Rating=Ratings Mapping

Plus 0.25 if negative outlook

Plus 0.35 if watch negative

Plus 0.00 if stable, developing, watch developing

Minus 0.25 if positive outlook

Minus 0.35 if watch positive

A snapshot of ratings for each credit research agency at the point ofany ratings change maybe taken since there could be multiple ratingschanges during a measurement period. For example, if there were twointernal company ratings changes and two different credit researchagencies changes during the same measurement period, the before andafter rating to the credit research agency ratings at the point ofchange would need to be compared. If always compared to the currentrating, the calculation may be comparing to the wrong rating. The datatable capturing the ratings changes may take a snapshot of the otherratings at the time so that they can be compared.

The third metric of the net impact may be calculated as described above.The fourth metric of graphical depictions then may be run. Illustrativeexamples include internal company ratings with respect to time, creditresearch agency ratings with respect to time, and difference in opinionbetween the two with respect to time. A graphical presentation ofratings by issuer and by credit research agency may also be shown. Afunction may be included in the database to show a pivot chart. Dropdown lists may be included to allow a user to pick an issuer and creditresearch agency in the pivot chart. Date may be on the x axis of thegraphical representations so that a history of ratings may be shown overtime. For difference of opinion data, a graphical presentation of themagnitude of the difference in ratings by a credit research agency maybe shown.

The fifth metric of percentage of times different may be run. Thispercentage may be a count of securities for which different ratingsexist divided by the total number of securities rated by the internalcompany. If the absolute value of the difference between the internalcompany rating, as calculated in percentage of times ahead andpercentage of times behind, is greater than zero, then there is adifference.

With respect to FIG. 4, steps 423, 425, and 427 correlate to step 309,create scorecard, from FIG. 3. In step 423, a scorecard may be populatedwith core metrics data. An illustrative scorecard follows:

Period: 2Q08 Analyst: Name No. of Ratings Changes: 18 No. of InternalCompany Ratings Changes: 13 First Agency's Ratings Changes: 11 SecondAgency's Ratings Changes: 7 Third Agency's Ratings Changes: 6 RightDirection: 9 Wrong Direction: 3 Flat Direction: 0 Ratings Effectiveness:82% % Ahead: 50% % Behind: 50% Net Impact: 0.3 bps % Times Different:63%

In step 425, qualitative analysis may be completed. For example,notations may be made that during the period, the first credit researchagency raised and lowered several bank ratings as a result of twochanges in their rating methodology. These actions led to multipleratings changes by the first credit research agency. Then, in step 427,the scorecard may be distributed to stakeholders.

While illustrative systems and methods as described herein embodyingvarious aspects of the present disclosure are shown, it will beunderstood by those skilled in the art, that the disclosure is notlimited to these embodiments. Modifications may be made by those skilledin the art, particularly in light of the foregoing teachings. Forexample, each of the elements of the aforementioned embodiments may beutilized alone or in combination or subcombination with elements of theother embodiments. It will also be appreciated and understood thatmodifications may be made without departing from the true spirit andscope of the present disclosure. The description is thus to be regardedas illustrative instead of restrictive on the present disclosure.

1. A computer implemented method comprising: determining, by a computer,an internal company credit rating for each of a plurality of securities;inputting the internal company credit rating for each of the pluralityof securities and an external credit research agency original creditrating for each of the securities in a ratings history database;receiving, by the computer, data representative of a change of theexternal credit research agency original credit rating for at least onesecurity of the plurality of securities to a new credit rating;responsive to the change, receiving, by the computer, datarepresentative of a current price of the at least one security and abenchmark price of the at least one security, the benchmark price beinga price of the at least one security at a designated period before thechange; and calculating, by the computer, at least one metric todetermine the number of times the internal company credit ratings foreach of the plurality of securities converged with the received datarepresentative of the change of the external credit research agencyoriginal credit rating, the calculating based upon the internal companycredit rating for the at least one security, the external creditresearch agency original credit rating for the at least one security,the external credit research agency new credit rating for the at leastone security, the current price of the at least one security, and thebenchmark price of the at least one security, wherein calculating the atleast one metric comprises determining a count of the number of timesthe absolute difference between the internal company credit rating forthe at least one security and the external credit research agency newcredit rating for the at least one security is less than the absolutedifference between the internal company credit rating for the at leastone security and the external credit research agency original creditrating for the at least one security.
 2. The computer implemented methodof claim 1, further comprising maintaining the current price and thebenchmark price of the at least one security and the external creditresearch agency new credit rating for the at least one security in theratings history database.
 3. The computer implemented method of claim 1,further comprising receiving data representative of a change to theinternal company credit rating for the at least one security.
 4. Thecomputer implemented method of claim 1, further comprising: for at leastone of the plurality of securities, determining whether the datarepresentative of the change of the external credit research agencyoriginal credit rating has converged with the associated internalcompany credit rating; and in response to determining that the datarepresentative of the change of the external credit research agencyoriginal credit rating has converged with the associated internalcompany credit rating, identifying the associated internal companycredit rating as a correct assessment.
 5. The computer implementedmethod of claim 4, further comprising crediting an analyst associatedwith establishing the internal company credit rating for the at leastone of the plurality of securities.
 6. The computer implemented methodof claim 5, wherein the external credit research agency new creditrating for the at least one security is higher than the external creditresearch agency original credit rating for the at least one security. 7.One or more non-transitory computer readable media storing computerexecutable instructions that, when executed by at least one processor,cause the at least one processor to perform a method comprising:receiving an internal company credit rating for each of a plurality ofsecurities; receiving an external credit research agency original creditrating for each of the securities in a ratings history database;receiving data representative of a change of the external creditresearch agency original credit rating for at least one security of theplurality of securities to a new credit rating; responsive to thechange, receiving data representative of a current price of the at leastone security and a benchmark price of the at least one security, thebenchmark price being a price of the at least one security at adesignated period before the change; and calculating, by the computer,at least one metric to determine the number of times the internalcompany credit ratings for each of the plurality of securities convergedwith the received data representative of the change of the externalcredit research agency original credit rating, the calculating basedupon the internal company credit rating for the at least one security,the external credit research agency original credit rating for the atleast one security, the external credit research agency new creditrating for the at least one security, the current price of the at leastone security, and the benchmark price of the at least one security,wherein calculating the at least one metric comprises determining acount of the number of times the absolute difference between theinternal company credit rating for the at least one security and theexternal credit research agency new credit rating for the at least onesecurity is less than the absolute difference between the internalcompany credit rating for the at least one security and the externalcredit research agency original credit rating for the at least onesecurity.
 8. The one or more non-transitory computer readable media ofclaim 7, the method further comprising maintaining the current price andthe benchmark price of the at least one security and the external creditresearch agency new credit rating for the at least one security in theratings history database.
 9. The one or more non-transitory computerreadable media of claim 8, wherein the external credit research agencynew credit rating for the at least one security is lower than theexternal credit research agency original credit rating for the at leastone security.
 10. The one or more non-transitory computer readable mediaof claim 7, the method further comprising receiving data representativeof a change to the internal company credit rating for the at least onesecurity.
 11. The one or more non-transitory computer readable media ofclaim 7, the method further comprising: for at least one of theplurality of securities, determining whether the data representative ofthe change of the external credit research agency original credit ratinghas converged with the associated internal company credit rating; and inresponse to determining that the data representative of the change ofthe external credit research agency original credit rating has convergedwith the associated internal company credit rating, identifying theassociated internal company credit rating as a correct assessment. 12.The one or more non-transitory computer readable media of claim 11, themethod further comprising crediting an analyst associated withestablishing the internal company credit rating for the at least one ofthe plurality of securities.
 13. The one or more non-transitory computerreadable media of claim 12, wherein the external credit research agencynew credit rating for the at least one security is higher than theexternal credit research agency original credit rating for the at leastone security.
 14. A system comprising: at least one database configuredto maintain an internal company credit rating for each of the pluralityof securities and an external credit research agency original creditrating for each of the securities; and at least one computing device,operatively connected to the at least one database, configured to:receive data representative of a change of the external credit researchagency original credit rating for at least one security of the pluralityof securities to a new credit rating; responsive to the change, receivedata representative of a current price of the at least one security anda benchmark price of the at least one security, the benchmark pricebeing a price of the at least one security at a designated period beforethe change; and calculate at least one metric to determine the number oftimes the internal company credit ratings for each of the plurality ofsecurities converged with the received data representative of the changeof the external credit research agency original credit rating, thecalculating based upon the internal company credit rating for the atleast one security, the external credit research agency original creditrating for the at least one security, the external credit researchagency new credit rating for the at least one security, the currentprice of the at least one security, and the benchmark price of the atleast one security, wherein the at least one computing device isconfigured to calculate the at least one metric based on a count of thenumber of times the absolute difference between the internal companycredit rating for the at least one security and the external creditresearch agency new credit rating for the at least one security is lessthan the absolute difference between the internal company credit ratingfor the at least one security and the external credit research agencyoriginal credit rating for the at least one security.
 15. The system ofclaim 14, the at least one computing device further configured toreceive data representative of a change to the internal company creditrating for the at least one security.
 16. The system of claim 14, the atleast one computing device further is configured to: for at least one ofthe plurality of securities, determine whether the data representativeof the change of the external credit research agency original creditrating has converged with the associated internal company credit rating;and in response to determining that the data representative of thechange of the external credit research agency original credit rating hasconverged with the associated internal company credit rating, identifythe associated internal company credit rating as a correct assessment.17. The system of claim 16, the at least one computing device further isconfigured to credit an analyst associated with establishing theinternal company credit rating for the at least one of the plurality ofsecurities.
 18. One or more non-transitory computer readable mediastoring computer executable instructions that, when executed by at leastone processor, cause the at least one processor to perform a methodcomprising: receiving an internal company credit rating for each of aplurality of securities; receiving an external credit research agencyoriginal credit rating for each of the securities in a ratings historydatabase; receiving data representative of a change of the externalcredit research agency original credit rating for at least one securityof the plurality of securities to a new credit rating; responsive to thechange, receiving data representative of a current price of the at leastone security and a benchmark price of the at least one security, thebenchmark price being a price of the at least one security at adesignated period before the change; and calculating, by a computer, atleast one metric to determine the number of times the internal companycredit ratings for each of the plurality of securities converged withthe received data representative of the change of the external creditresearch agency original credit rating, wherein calculating the at leastone metric comprises determining a count of the number of times theabsolute difference between the internal company credit rating for theat least one security and the external credit research agency new creditrating for the at least one security is less than the absolutedifference between the internal company credit rating for the at leastone security and the external credit research agency original creditrating for the at least one security.
 19. The one or more non-transitorycomputer readable media of claim 18, the method further comprisingcalculating, by the computer, at least one metric to determine thenumber of times the internal company credit ratings for each of theplurality of securities converged ahead of time.
 20. The one or morenon-transitory computer readable media of claim 18, the method furthercomprising calculating, by the computer, at least one metric todetermine the number of times the internal company credit ratings foreach of the plurality of securities converged behind the time.